individual skill
- Asia > Middle East > Jordan (0.04)
- Asia > China > Anhui Province > Hefei (0.04)
- Asia > Middle East > Jordan (0.04)
- Asia > China > Anhui Province > Hefei (0.04)
A Theory for Emergence of Complex Skills in Language Models
Arora, Sanjeev, Goyal, Anirudh
A major driver of AI products today is the fact that new skills emerge in language models when their parameter set and training corpora are scaled up. This phenomenon is poorly understood, and a mechanistic explanation via mathematical analysis of gradient-based training seems difficult. The current paper takes a different approach, analysing emergence using the famous (and empirical) Scaling Laws of LLMs and a simple statistical framework. Contributions include: (a) A statistical framework that relates cross-entropy loss of LLMs to competence on the basic skills that underlie language tasks. (b) Mathematical analysis showing that the Scaling Laws imply a strong form of inductive bias that allows the pre-trained model to learn very efficiently. We informally call this {\em slingshot generalization} since naively viewed it appears to give competence levels at skills that violate usual generalization theory. (c) A key example of slingshot generalization, that competence at executing tasks involving $k$-tuples of skills emerges essentially at the same scaling and same rate as competence on the elementary skills themselves.
- North America > United States > Massachusetts (0.04)
- Europe > United Kingdom > England > Oxfordshire > Oxford (0.04)
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.04)
- Asia > Middle East > Jordan (0.04)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Grammars & Parsing (0.93)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.93)
A Generalized Bradley-Terry Model: From Group Competition to Individual Skill
The Bradley-Terry model for paired comparison has been popular in many areas. We propose a generalized version in which paired individual comparisons are extended to paired team comparisons. We introduce a simple algorithm with convergence proofs to solve the model and obtain individual skill. A useful application to multi-class probability estimates using error-correcting codes is demonstrated.
How AI can help choose your next career and stay ahead of automation
The typical Australian will change careers five to seven times during their professional lifetime, by some estimates. And this is likely to increase as new technologies automate labor, production is moved abroad, and economic crises unfold. Jobs disappearing is not a new phenomenon--have you seen an elevator operator recently? New technologies also create new jobs, but the skills they require do not always match the old jobs. Successfully moving between jobs requires making the most of your current skills and acquiring new ones, but these transitions can falter if the gap between old and new skills is too large.
- Health & Medicine (0.39)
- Banking & Finance > Economy (0.36)
Females boost collective intelligence more than men, study finds
Having more women in a team or group can boost the overall'collective intelligence' for decision making, when compared to a male dominated group, study reveals. Researchers from Pennsylvania's Carnegie Mellon University examined 22 studies covering 5,349 individuals engaged in group and individual activities. The team found that individual skill, group gender composition, and group collaboration were all predictors of collective intelligence, or the ability of a group to work together and solve a range of problems that vary in complexity. The research, that involved running machine learning algorithms over multiple large sets of data, revealed that the success of a group activity could be better predicted using collective intelligence measures than on individual member skill. These collective measures included social perceptiveness of individual members, group composition, particularly female proportion and age diversity, and group size.
Brain-inspired algorithm helps AI systems multitask and remember
Behind most of today's artificial intelligence technologies, from self-driving cars to facial recognition and virtual assistants, lie artificial neural networks. Though based loosely on the way neurons communicate in the brain, these "deep learning" systems remain incapable of many basic functions that would be essential for primates and other organisms. However, a new study from University of Chicago neuroscientists found that adapting a well-known brain mechanism can dramatically improve the ability of artificial neural networks to learn multiple tasks and avoid the persistent AI challenge of "catastrophic forgetting." The study, published in Proceedings of the National Academy of Sciences, provides a unique example of how neuroscience research can inform new computer science strategies, and, conversely, how AI technology can help scientists better understand the human brain. When combined with previously reported methods for stabilizing synaptic connections in artificial neural networks, the new algorithm allowed single artificial neural networks to learn and perform hundreds of tasks with only minimal loss of accuracy, potentially enabling more powerful and efficient AI technologies.